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DETAILED ACTION 
Information Disclosure Statement 

1. The examiner has considered the IDS filed December 20, 2002 and an initialed copy is 
included with this office action. However, the reference to Panoptic website has not been 
considered because a relevant date is not identified for the particular reference. Other website 
references have a date on which the reference was printed. 

Claim Objections 

2. Claim 33 recites the feature of "imitating sensing with a switch movable relative to the 
second location" does this mean that the user places the switch somewhere and the user then 
pretends to be sensing? Or its possibly a spelling error and "imitating" should be -initiating-. 
Until the applicant responds, the examiner will interpret "imitating" to be —initiating—. 

Claim Rejections - 35 USC § 112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

4. Claim 35 is rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Claim 35 recites the limitation "positioning" in line 1 of the claim. There is insufficient 
antecedent basis for this limitation in the claim or claims 34, 31, 30 or 27. 

Claim Rejections - 35 USC §102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
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A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) and the Intellectual Property and High Technology Technical Amendments Act of 2002 
do not apply when the reference is a U.S. patent resulting directly or indirectly from an 
international application filed before November 29, 2000. Therefore, the prior art date of the 
reference is determined under 35 U.S.C. 102(e) prior to the amendment by the AIPA (pre-AIPA 
35 U.S.C. 102(e)). 

6. Claims 1, 2 and 4 are rejected under 35 U.S.C. 102(b) as being clearly anticipated by 
Saund, US 5528290. 

As per claim 1, Saund teaches: 

a visual sensor (camera, col. 3, 11. 12-13) providing image data corresponding to sensed 
images, the visual sensor selectively directed toward a first position to sense a first image (col. 3, 
11. 22-23) and toward a second position to sense a second image (col. 3, 11. 22-23 and fig. 3); and 

an image processor (which performs "center surround processing") coupled to the visual 
sensor to receive the image data from the visual sensor, the image processor capable of 
processing the image data as a function of direction( based upon a gradient operator, col. 4, 11. 
10-15) of the visual sensor toward the first position or the second position, (col. 4, 11. 64-67, 
where the dead reckoning is based on position to pan/tilt of camera when image is captured). 

As per claim 2, Saund teaches: 
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a storage device (memory, col. 3, 11. 14-15) for storing a first processing value and a 
second processing value (computational images, col. 3, 1. 15), wherein the image processor 
processes the image data of the first position using the first processing value (fig. 5, step 130), 
and wherein the image processor processes the image data of the second position using the 
second processing value (fig. 5, step 132). 

As per claim 4, Saund teaches: 

wherein the first processing value relates to an optical correction for distortion of the first 
image (fig. 2, step 102 and 106) and the second processing value relates to an optical correction 
for distortion of the second image (fig. 2, step 108). Saund teaches that all tiles are perspective 
corrected based upon a perspective distortion correction transformation that is calculated the first 
time and updated as the system is used (col. 5, 11. 54-60). 

7. Claims 7, 18, 26, 27 and 30 are rejected under 35 U.S.C 102(e) as being clearly 
anticipated by Oliver, US 6330082 B 1 . 
As per claim 7, Oliver teaches: 

a visual sensor (scanner-converter, fig. 5, element 111) adapted to provide image data 
corresponding to sensed visual images of a writing surface (whiteboard, fig. 5, element 1 14); and 

an image processor (converter) coupled to the visual sensor to receive the image data 
from the visual sensor, the image processor capable of identifying information provided on the 
writing surface apart from the writing surface (col. 5, 11. 17-21). 
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Note that the examiner is interpreting "identifying" as meaning that the system merely 
recreates the data written on the whiteboard as opposed to a blank whiteboard by reproducing the 
image data on a display screen or an associated printer. 

As per claim 18, Oliver teaches: 

In combination with a defined writing surface provided in a fixed location in a room 
(Fig. 5, element 14), an image capturing system disposed in the room at a second location remote 
from the writing surface (fig. 5, element 1 1 1) to sense successive visual images of the writing 
surface (col. 6, 11. 18-21) and adapted to identify information provided on the writing surface 
(col. 5, 11. 17-21, identification is performed by recreating the image data on a display screen or 
an associated printer). 

As per claim 26, Oliver teaches: 

wherein the image capturing system includes a visual sensor adapted to scan the writing 
surface, (fig. 5, col. 5, 11. 13-20). 

As per claim 27, Oliver teaches: 

locating an image capturing system at a second location in the room remote from the 
writing surface (Fig. 5, element 14); 

sensing a visual image of the writing surface with the image capturing system (fig. 5, 
element 111, col. 6, 11. 18-21); and 

identifying information provided on the writing surface with the image capturing system, 
(col. 5, 11. 17-21, identification is performed by recreating the image data on a display screen or 
an associated printer). 

As per claim 30, Oliver teaches: 
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wherein sensing includes sensing a plurality of visual images (line images, col. 5, 11. 16- 

17). 

8. Claims 13 and 14 are rejected under 35 U.S.C. 102(b) as being clearly anticipated by 
Shimizu et al., (hereinafter Shimizu), US 5450127. 
As per claim 13, Shimizu teaches: 

a visual sensor providing image data corresponding to sensed images, the visual sensor 
comprising a set of adjacent sensing elements being exposed collectively to successive portions 
of the image (figures 1 and 4, col. 3, 11. 34-38); and 

a storage device for storing sensing element control values (Reference level (step S20), 
col. 6, 11. 34-35, inherently must be stored in some form of memory for it to be considered a 
reference level.) 

a controller coupled to the storage device and the visual sensor (fig. 3, element 10), the 
controller controlling a time duration of exposure of the sensing elements (col. 6, 11. 30-35) to the 
portion of image as a function of exposure to successive portions (fig. 6, col. 6, 11. 50-60). 

As per claim 14, Shimizu teaches: 

wherein a sensing element control value is provided for each sensing element for each 
successive portion of the image (col. 4, 1. 67-col. 5, 1. 2). Each element of the line CCD is 
ultimately stored as a pixel value so as to provide uniform overexposure protection or 
underexposure correction the control value must be used for each element of the line CCD as the 
element values are readout of the CCD. 
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Claim Rejections - 35 USC § 103 



9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

10. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Saund, as applied 
to claim 1 above and further in view of Tatsumi et al, (hereinafter Tatsumi), US 6476862 Bl. 

As per claim 3, Suand teaches a zoomed- in image, col. 3, 11. 19, and camera calibration 
settings are used when images are captured, col. 4, 11. 65-66. But Saund does not go into further 
detail, however, Tatsumi teaches: 

wherein the visual sensor includes a zoom lens (figure 1, element 1, col. 43-48), and 
wherein the first processing value relates to a first setting of the zoom lens and the second 
processing value relates to a second setting of the zoom lens (col. 7, 1. 65-col. 8, 1. 5). 

It would have been obvious to one of ordinary skill in the art to use the predetermined 
zoom settings of Tatsumi as inputs into the perspective transformations of Saund to reduce the 



need for further compensation by the transformation or to account for the additional distance that 
the subject matter being imaged is away from the camera. 

1 1 . Claims 5 and 6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Saund, as 
applied to claim 1 above and further in view of Oliver, US 6330082. 

As per claim 5, Saund teaches using a camera but provides no details, however, Oliver 
teaches: 
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wherein the visual sensor comprises a sensing device having a plurality of sensing 
elements (col. 1, 11. 13-16, and linear photosensor, col. 5, 1. 11, typically linear photo sensors are 
an array of CCDs). 

It would have been obvious to one of ordinary skill in the art to use the scanner converter 
of Oliver in the system of Saund to insure that sequential portions of the tiled image has the 
proper data values before a high speed scan is initiated. 

As per claim 6, Oliver, teaches: 

wherein the sensing device comprises a linear array of sensing elements (col. 5, 11 16- 

21). 

12. Claims 8, 9, 11, 12, 19, 28, 31, 32, and 37- 39 are rejected under 35 U.S. C. 103(a) as 
being unpatentable over Oliver, as applied to claims 1 and 27 above and further in view of Hong 
et al, (hereinafter Hong), US 5764799. 

As per claim 8, Oliver teaches that his system can scan an image of a whiteboard (col. 5, 
11. 7-10), but Oliver does not specifically state what type of processing is accomplished after 
collecting the image. Hong also scans an image, fig. 3, element 301, on a writing surface. 
However, Hong teaches: 

comprising a storage device (RAM, col. 4, 1. 11) for storing a reference visual image (col. 
4, 11. 17-19), and wherein the image processor is coupled to the storage device to access the 
reference visual image to identify information provided on the writing surface (col. 4, 11. 1 1-20, 
col. 10, 11. 43-50). 
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It would have been obvious to one of ordinary skill in the art to use the optical character 
recognition methods of Hong with the remote scene scanner of Oliver to provide a system that 
can provide digitized data with respect to the scene scanned as in figure 5 of Oliver. In addition, 
Hong merely needs "any suitable image digitizer" (col. 3, 11. 51-52) to provide signals to the 
optical character recognition circuit. 

As per claim 9, Oliver teaches: 

wherein the visual sensor comprises a sensing device adapted to scan the writing surface 
(fig. 5, element 111). 

As per claim 1 1, Oliver teaches: 

wherein the image processor is adapted to identify an area requiring reimaging (col. 8, 11. 

38-43). 

As per claim 12, Oliver teaches: 

wherein the image processor controls the visual sensor to obtain a second visual image of 
at least the area (scan line portions, col. 5, 11. 15-17), if reimaging is required, and wherein the 
processor is adapted to combine the first-mentioned visual image with the second visual image 
(col. 5, 11. 19-20). 

As per claim 39, it recites substantially the same limitations as claim 12 above and 
analogous remarks apply. 

As per claim 19, Hong teaches: 

wherein the image capturing system including an image processor to identify information 
on the writing surface as a function of a reference visual image of the writing surface, (col. 4, 11. 
11-20, col. 10, 11. 43-50). 
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As per claim 28, Oliver does not specifically teach that the identifying step is performed 
as a function of a reference image. However, Hong teaches: 

wherein identifying information includes identifying information as a function of a 
reference visual image of the writing surface, (col. 4, 11. 1 1-20, col. 10, 11. 43-50). 

As per claim 3 1, Hong teaches: 

wherein identifying includes identifying information as a function of a plurality of visual 
images (After Oliver collects the line images via CCD as does Hong, the image data must be 
processed for Optical Character Recognition, col. 3, 11. 50-56, a complete set of data is a function 
of the line images collected by the CCD). 

As per claim 32, Oliver does not teach optical character recognition. However, Hong 
teaches: 

wherein identifying includes identifying includes comparing a first visual image to a 
second visual image, (col. 4, 11. 37-43 or fig. 2, col. 4, 11. 46-50). 
As per claim 37, both Oliver and Hong teach: 

wherein sensing comprises scanning the writing surface (Oliver, fig. 5, element 1 14, 
Hong, fig. 3, element 303). 

As per claim 38, Oliver teaches: 

wherein identifying information includes detecting an area of the writing surface 
requiring reimaging. (col. 8, 11. 38-43). 

13. Claims 33-35 are rejected under 35 U.S.C. 103(a) as being unpatentable over Oliver and 
Hong, as applied to claim 31 above and further in view of Inagaki, US 5999214. 
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Inagaki teaches a video conference system that includes a video camera for imaging 
participants, a plotting image input unit in a single system (Abstract). 

As per claim 33, neither Oliver nor Hong teaches a switch movable relative to a second 
location. However, Inagaki teaches: 

wherein sensing a visual image includes [imitating] initiating (remote switch responsive 
to voice or manual input, col. 12, 11. 4-10) sensing of a visual image with a switch movable 
relative to the second location (col. 12, 11. 1 1-25). 

It would have been obvious to one of ordinary skill in the art to use the voice direction 
detection feature of Inagaki in the system of Oliver and Hong to allow for imaging of both a 
whiteboard surface (or plotting surface) and a person speaking without having to rely upon 
another person to direct the imaging process and to reliably image the person speaking (col. 12, 
11. 24-25). 

As per claim 34, neither Oliver nor Hong teaches a selectively directable imaging system. 
However, Inagaki teaches: 

wherein the image capturing system is selectively directable (col. 1, 11. 25-31) to obtain a 
visual image of the writing surface (plotting device which is equivalent to a whiteboard as taught 
by Oliver (col. 5, 11. 7-10)) and a visual image of a second area (another participant, Inagaki, col. 
2, 11. 19-23 as in figs. 1A and IB) the method further comprising directing the image capture 
system toward the writing surface or the second area (participants or plotting device, (col. 3, 11. 
5-10), and wherein sensing includes sensing a visual image of the writing surface or the second 
area (col. 3, 11. 10-20). The prior art system as explained can either display a single participant or 
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another input device and since as claimed the image capture system does not consist of one 
image capture device, the plurality of cameras taught by Inagaki teaches the claimed invention. 

It would have been obvious to one of ordinary skill in the art to use the features of 
Inagaki's video conference system with the writing surface imagers of Oliver and Hong to 
provide the superimposing text that was written by the participants and to improve the ambience 
of the conference and to improve the search for a participant that is speaking. 

As per claim 35, Inagaki teaches: 

wherein positioning includes operating a switch to direct the image capture system 
toward the writing surface or the second area (col. 2, 11. 19-23). 

14. Claim 36 rejected under 35 U.S. C. 103(a) as being unpatentable over Oliver, Hong, and 
Inagaki, as applied to claim 34 above and further in view of Saund. 

As per claim 36, neither Oliver Hong nor Inagaki teaches having a first and second 
processing value, however, Saund teaches: 

wherein sensing comprises providing image data corresponding to the visual image 
(tiles), and wherein the method further comprises storing a first processing value (transformation 
value for a first pixel, fig. 4) and a second processing value (transformation value for a second 
pixel), and processing the image data using the first processing value and the second processing 
value as a function of direction (location of the tile with respect to the camera, col. 4, 11. 62-67) 
of the image capture system toward the writing surface or the second area, respectively (col. 3, 11. 
54-65 or col. 5, 11. 46-51). 
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It would have been obvious to one of ordinary skill in the art to use the distortion 
compensation of Saund in the system of Oliver, Hong, and Inagaki to provide for a system that 
will allow for distortion correction when the to compensate for any distortions due to off-axis 
viewpoints of the camera and uneven lightness levels across the board due to uncontrolled 
lighting conditions or reflections (Saund, col. 3, 11. 32-35). 



15. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Shimizu, as applied to claim 13 above and further in view of Hong et al., (hereinafter Hong), US 
5764799. 

As per claim 15, Shimizu teaches that his system can scan an image of a blackboard (col 
1, 11. 10-13), but Shimizu does not specifically state what type of processing is accomplished 
after collecting the image. Hong also scans an image, fig. 3, element 301, on a writing surface. 
However, Hong teaches: 

an image processor coupled to the visual sensor to receive the image data from the visual 
sensor (col. 3, 11. 50-56), the image processor capable of identifying information provided on the 
writing surface apart, from the writing surface, (col. 4, 11. 11-20, 27-31, col. 10, 11. 43-50). 

It would have been obvious to one of ordinary skill in the art to use the optical character 
recognition methods of Hong with the remote scene scanner of Shimizu to provide a system that 
can provide digitized data with respect to a scanned blackboard. In addition, Hong merely needs 
"any suitable image digitizer" (col. 3, 11. 51-52) to provide signals to the optical character 
recognition circuit. 
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As per claim 16, Hong teaches: 

comprising a storage device for storing a reference visual image (col. 4, 11. 11-16), and 
wherein the image processor is coupled to the storage device to access the reference visual image 
(col. 4, 11. 23-28) to identify information provided on the writing surface. 

16. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shimizu and 
Hong, as applied to claim 15 above and further in view of Saund. 

Neither Shimizu nor Hong teach correcting for optical distortion. However, Saund 
teaches: 

a storage device (memory, col. 3, 11. 14-15) for storing a processing value related to 
correction of the visual image for optical distortion (figs. 2 and 4), wherein the image processor 
is coupled to the storage device to access and use the processing value during image processing, 
(col. 3, 11. 54-65 or col. 5, 11. 46-51). 

It would have been obvious to one of ordinary skill in the art to use the perspective 
distortion compensation transformations of Saund in the imaging system of Shimizu and Hong to 
compensate for any distortions due to off-axis viewpoints of the imager and uneven lightness 
levels across the board due to uncontrolled lighting conditions or reflections (Saund, col. 3, 11. 
32-35). 

17. Claims 10, 20, 21 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Oliver, as applied to claim 1 above and further in view of Saund. 
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As per claim 10, Oliver does not teach compensating for distortion. However, Saund 
teaches: 

a storage device (memory, col. 3, 11. 14-15) for storing a processing value related to 
correction of the visual image for optical distortion (figs. 2 and 4) wherein the image processor is 
coupled to the storage device to access and use the processing value during image processing, 
(col. 3, 11. 54-65 or col. 5, 11. 46-51). 

It would have been obvious to one of ordinary skill in the art to use the perspective 
distortion compensation transformations of Saund in the imaging system of Oliver to compensate 
for any distortions due to off-axis viewpoints of the camera and uneven lightness levels across 
the board due to uncontrolled lighting conditions or reflections (Saund, col. 3, 11 32-35). 

As per claim 20, Saund teaches: 

wherein the image capturing system includes a visual sensor disposed above the writing 
surface, (fig. 1, element 54, the writing surface is on the wall and the visual sensor is located 
above the wall on the ceiling). 

It would have been obvious to one of ordinary skill in the art to mount the remote scene 
imager of Oliver (col. 1 , 11. 6-9) to the ceiling to provide more efficient use of room space as 
shown in Suand (figure 1). Spectators could sit beneath and in front of the scanner without 
obstructing the scanners view of the whiteboard. It would have been particularly obvious to one 
of ordinary skill in the art that if the system can be used on a floor that it could also be mounted 
above the scene to be imaged. 

As per claim 21, Saund teaches: 

wherein the visual sensor is mounted to a ceiling of the room. (fig. 1, element 54). 
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As per claim 29, Oliver does not teach compensating for distortion. However, Saund 
teaches compensating for perspective distortion; therefore, Saund teaches: 

wherein sensing a visual image includes compensating for distortion of the visual image, 
(col. 3, 11. 30-33, col. 4, 1. 62-col. 5, 1. 8, col. 5, 11. 46-52). 

It would have been obvious to one of ordinary skill in the art to use the distortion 
correction features of Saund when there may be a slight misalignment of the line images of the 
scanning performed by Oliver. 

18. Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over Oliver, as applied 
to claim 18 above and further in view of Inagaki. 

As per claim 22, Oliver teaches that his scanner can be located remotely from a scene to 
be imaged (col. 5, 11. 7-10), but Oliver does not teach imaging another part of a room. However, 
Inagaki teaches: 

wherein the image capturing system includes a visual sensor disposed within the room to 
sense images of the writing surface (plotting device, figure 12, element 16) and a second area 
spaced apart (figs. 4A-4D and fig. 12, element 110) from the writing surface. 

It would have been obvious to one of ordinary skill in the art to use additional imaging 
capabilities of Inagaki including the suggestion of having a plotting device imager to allow for 
data to be input into the video conference system of Inagaki to allow for reliable switching 
between inputting image data from a whiteboard while maintaining video coverage of the person 
speaking during a video conference (col. 12, 11. 23-25). 
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19. Claims 23-25 are rejected under 35 U.S.C 103(a) as being unpatentable over Oliver, and 
Inagaki, as applied to claim 22 above and further in view of Saund. 

As per claim 23 , although Oliver teaches that his camera can be remotely located from 
the image being scanned, Oliver only gives one example of where his scene scanner can be 
located, see fig. 5. Inagaki also does not specifically state where his imager can be located 
within a room. However, Saund teaches: 

wherein the visual sensor is mounted to a wall of the room. See fig. 1 . Saund teaches that 
the system can be located on the ceiling in fig. 1, but mounting the camera high on the wall 
serves as the functional equivalent of mounting the camera on a wall. Therefore, it would have 
been obvious to one of ordinary skill in the art to mount the imaging device on a wall because 
neither Oliver nor Inagaki limits the user on where to locate the imaging device and the Saund 
suggests that an alternative location and equally functional location of the imager could be on the 
ceiling. Also, Inagaki teaches that his system has a remote panhead that allows for panning of 
the camera (col. 1, 1. 27). 

As per claim 24, Saund teaches: 

wherein the visual sensor is disposed above the writing surface (fig. 1). 
As per claim 25, Saund teaches: 

wherein the visual sensor is mounted to a ceiling of the room. (fig. 1). 



Conclusion 

20. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 
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The following U.S. patent(s) and technical disclosure refer(s) to reimaging: Neville, 
3818445, Kumar et al, 6522787, Smith, 6539112, and IBM Technical Disclosure Bulletin, 
NN7007351, July 1970. 

The following U.S. patent(s) refer(s) to document and/or room imagers: Yamane, 
5978028 and 6300975, Tanno et al., 6195176, Cass, 5581637, and Dance et al, 6512539. 

The following U.S. patent(s) refer(s) to videoconference systems: Lukacs, 573701 1, and 
Shibata et al, 5689300. 

The following U.S. patent(s) refer(s) to changing image perspective: Gallagher, 
6400848. 

The following Japanese publication refers to electronic blackboards: Hideki, 62-160868. 
21 . Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Martin Miller whose telephone number is (703) 306-9134. The 
examiner can normally be reached on Monday-Friday, 9am- 5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 872-93 14 for regular 
communications and (703) 872-9314 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (V02) 305-3900. 




4MEUAOU 
SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 260C 
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